Focus state display apparatus and focus state display method

ABSTRACT

A focus state display apparatus comprising focus area extraction means for extracting the image signals of a predetermined area from photographed image signals, edge enhancement processing means for enhancing the edge of the extracted image signals, time integration value calculation means for calculating an integration value of the edge-enhanced image signals in a certain period of time, focus state determination means for determining the focus state of the photographed image signals on the basis of the calculated integration value, and focus state display means for displaying the determined focus state. A user is capable of readily determining the focus state of a camera and confirming and adjusting the focus thereof with accuracy even in a display apparatus of a camera-equipped portable terminal device, where the size and resolution thereof are limited.

This application is a Divisional of co-pending application Ser. No.10/544,727 filed on Aug. 5, 2005, and for which priority is claimedunder 35 U.S.C. §120; the entire contents of all are hereby incorporatedby reference.

TECHNICAL FIELD

The present invention relates to an apparatus for displaying the focusstate of a camera and a display method thereof, especially to atechnique for displaying the focus state of a small portable camera anda fixed-focus camera disposed in a camera-equipped portable terminaldevice, such as a portable telephone equipped with an imaging function.

BACKGROUND ART

In recent years, with the development of image sensors having small sizeand low power consumption, portable devices with a built-in camera, suchas PDAs (Personal Digital Assistants), portable telephones, and portablePCs (personal computers) have been in widespread use.

These portable devices are capable of photographing images and printedmaterials instead of taking notes, for example, using a camera such as abuilt-in CCD camera. Also, they are capable of storing such photographedimages in the portable devices and transmitting the photographed imagesto other portable devices and PCs through the Internet via an electronicmail function of the portable devices.

Cameras built in such portable devices are manufactured such that theyare as small and light in weight as possible. In general, cameras havingresolution lower than that of normal digital cameras, namely, about 100thousand to 300 thousand effective pixels are used.

Also, cameras built in portable terminal devices involve severelimitation in terms of a case size and costs, so that, in general, anautomatic focus device is not built in. When a user photographs an imageusing such a built-in camera, an image in the photographing range of thebuilt-in camera is displayed on a display screen, such as an LCD (liquidcrystal display) disposed in a portable terminal device. Thus, it ispossible to photograph adjusting the focus while confirming the displayscreen.

In order to adjust the focus upon photographing, although the focusingmay be performed while the user confirms the display screen as mentionedabove, the focusing may also be performed such that a focus state isdisplayed by the portable terminal device, while the user confirms thedisplay of the focus state as well as the display screen. The portableterminal device can comprise a function for displaying the focus stateon the basis of the signal components of an image received using thebuilt-in camera.

An imaging apparatus described in Patent Document 1 (JP PatentPublication (Kokai) No. 3-11884 A (1991)) relates to an imagingapparatus such as a video camera having a viewfinder and focus controlmeans. The apparatus comprises a focus state display apparatus fordisplaying a focus state upon manual focusing based on the ratio of highfrequency components to low frequency components of photographed imagesignals.

A focus state display apparatus and the microscope apparatus describedin Patent Document 2 (JP Patent Publication (Kokai) No. 2002-196225 A)comprises a focus state display apparatus by which the high-frequencycomponents of spatial frequencies of image data are extracted togenerate differential image data, coloration is changed in accordancewith pixel values indicating the high-frequency components of the data,and a focus state is notified using the difference of color.

DISCLOSURE OF THE INVENTION

A display apparatus such as an LCD disposed in such a camera-equippedportable terminal device is small and has a small number of pixels, sothat a displayed image is coarse. Thus, it has been difficult to focuswhile determining a blur of a photographing image, for example, with thenaked eyes. Also, even the focus state display apparatus as described inPatent Document 1 (JP Patent Publication (Kokai) No. 3-11884 A (1991))and Patent Document 2 (JP Patent Publication (Kokai) No. 2002-196225 A)are provided, a displayed focus state is difficult for the user tounderstand and sometimes it is difficult to adjust the focus.

The present invention is made in view of such problems. It is an objectof the present invention to provide a focus state display apparatus anda focus state display method that allows the user to readily determinethe focus state of a camera and to confirm and adjust the focus in aprecise manner even in a display apparatus of a camera-equipped portableterminal device, where the size and resolution thereof are limited.

The present invention provides a focus state display apparatuscomprising focus area extraction means for extracting the image signalsof a predetermined area from photographed image signals, edgeenhancement processing means for enhancing the edge of the extractedimage signals, time integration value calculation means for calculatingan integration value of the edge-enhanced image signals in a certainperiod of time, focus state determination means for determining thefocus state of the photographed image signals on the basis of thecalculated integration value, and focus state display means fordisplaying the determined focus state.

The present invention also provides a focus state display apparatuscomprising focus area extraction means for extracting the image signalsof a certain area from photographed image signals, edge enhancementprocessing means for enhancing the edge of the extracted image signals,histogram preparation means for preparing a histogram of theedge-enhanced image signals, variance value calculation means forcalculating a variance value regarding those image signals not less thana certain threshold value in the prepared histogram, time integrationvalue calculation means for calculating an integration value of thecalculated variance value of the image signals in a certain period oftime, focus state determination means for determining the focus state ofthe photographed image signals on the basis of the calculatedintegration value, and focus state display means for displaying thedetermined focus state.

In the focus state display apparatus of the present invention, the focusstate display means is characterized in that it displays in aquantitative or qualitative manner. In this case, quantitative displayinvolves displaying the magnitude of the value of the focus statecalculated in the focus state display apparatus, where as the valuebecomes larger, the focus is more correctly adjusted. Also, qualitativedisplay involves varying the form of display depending on the magnitudeof the value of the aforementioned focus state.

This enables displaying the focus state in various forms that arereadily understandable to the user. The user is capable of adjusting thefocus of a photographing image, moving and operating an imagingapparatus while viewing the display of the focus state.

In the focus state display apparatus of the present invention, examplesof quantitative display of the focus state via the focus state displaymeans include forms where the focus state of the photographed imagesignals is displayed using a bar graph, a level meter, numerical values,the number of graphic forms, and the like.

Further, when performing the quantitative display of the aforementionedfocus state, it is preferable to display the maximum value of the focusstate of the photographed image signals from the start of photographingto the present time along with the current focus state.

By contrast, in the focus state display apparatus of the presentinvention, examples of qualitative display of the focus state via thefocus state display means include forms below.

A form of the focus state display apparatus further comprising lightemitting means, where the focus state display means displays the focusstate of the photographed image signals by blinking or lighting thelight emitting means.

A form of the focus state display apparatus further comprisingsound-producing means for uttering a sound or a voice, where the focusstate display means displays the focus state of the photographed imagesignals by causing the sound-producing means to produce a sound or avoice.

A form of the focus state display apparatus further comprising filtermeans for removing the high-frequency components of the spatialfrequency components of the photographed image signals, where the focusstate display means modifies the range of spatial frequency componentsto be removed from the image signals via the filter means, depending onthe focus state of the photographed image signals.

A form where the focus state display means displays the focus state ofthe photographed image signals using the colors of a display image, thedensity of display characters, and the like.

Further, the present invention provides a camera comprising imagingmeans for photographing an image, image display means for displaying theimage photographed via the imaging means, and any one of theaforementioned focus state display apparatuses, and a portable terminaldevice having the camera.

The present invention also provides a focus state display methodincluding the steps of extracting the image signals of a predeterminedarea from photographed image signals, enhancing the edge of theextracted image signals, calculating an integration value of theedge-enhanced image signals in a certain period of time, determining thefocus state of the photographed image signals on the basis of thecalculated integration value, and displaying the determined focus state.

The present invention also provides a focus state display methodincluding the steps of extracting the image signals of a certain areafrom photographed image signals, enhancing the edge of the extractedimage signals, preparing a histogram of the edge-enhanced image signals,calculating a variance value regarding those image signals not less thana certain threshold value in the prepared histogram, calculating anintegration value of the calculated variance value of the image signalsin a certain period of time, determining the focus state of thephotographed image signals on the basis of the calculated integrationvalue, and displaying the determined focus state.

The present invention also provides a program including the steps ofextracting the image signals of a predetermined area from photographedimage signals, enhancing the edge of the extracted image signals,calculating an integration value of the edge-enhanced image signals in acertain period of time, determining the focus state of the photographedimage signals on the basis of the calculated integration value, anddisplaying the determined focus state.

The present invention also provides a program including the steps ofextracting the image signals of a certain area from photographed imagesignals, enhancing the edge of the extracted image signals, preparing ahistogram of the edge-enhanced image signals, calculating a variancevalue regarding those image signals not less than a certain thresholdvalue in the prepared histogram, calculating an integration value of thecalculated variance value of the image signals in a certain period oftime, determining the focus state of the photographed image signals onthe basis of the calculated integration value, and displaying thedetermined focus state.

The present invention further provides a recording medium in whicheither of the aforementioned programs is recorded.

Moreover, the present invention provides a focus state determinationapparatus comprising edge detection means for detecting the edge ofphotographed image signals, time integration value calculation means forcalculating an integration value of the difference of adjacent imagesignals in a certain period of time, the edge detected via the edgedetection means forming a boundary, and focus state determination meansfor determining the focus state of the photographed image signals on thebasis of the integration value calculated via the time integration valuecalculation means.

According to the focus state determination apparatus, it is possible toreadily determine the focus state by merely time-integrating thedifference of image signals such as the difference of luminance in anadjacent area with the detected edge forming a boundary. Especially, ina case where photographing targets are character codes, there is littlenecessity for performing edge enhancement, so that the focus state canbe determined by the aforementioned edge detection alone.

Moreover, there is provided a focus state determination apparatuscomprising histogram preparation means for preparing a histogram ofphotographed image signals, variance value calculation means forcalculating a variance value regarding those image signals not less thana certain threshold value in the prepared histogram, and focus statedetermination means for determining the focus state of the photographedimage signals on the basis of the calculated variance value of the imagesignals.

According to the aforementioned focus state determination apparatus,only those signals not less than a certain threshold value in thehistogram are used for the focus determination. Thus, edges generateddue to high frequency noises, hand movement of the user, or the like canbe removed and the influence thereof can be controlled.

There is provided a focus state determination apparatus having focusstate determination means for determining the focus state of charactersor a bar code in a case where photographing target switching meanscapable of switching between a normal mode handling general images asphotographing targets and an OCR function mode handling characters orbar codes as photographing targets switches to the OCR function mode,and a control portion for controlling such that it starts coderecognition means for recognizing photographed characters or bar codes.The apparatus may be constituted such that it starts in an automaticmanner when photographing targets are switched. Further, the apparatusmay have determination means for determining whether the photographingtargets are general images, characters, or bar codes.

The inventors have thought of disposing focus state display means in thefocus state display apparatus, the focus state display means determiningthe focus state of photographed image signals and performing visual oracoustic display that is readily understandable for the user inaccordance with the determined focus state.

In the following, embodiments of the present invention are described indetail with reference to the attached drawings.

FIGS. 1 to 13 show an example of the embodiments of the focus statedisplay apparatus and the focus state display method according to thepresent invention. In the drawings, portions provided with the samereference represent the same element and the basic constitution thereofis the same.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an outward appearance of a camera-equipped portableterminal device comprising a focus state display apparatus as a firstembodiment of the present invention.

FIG. 2 shows a functional block diagram indicating the constitution of afocus state display apparatus disposed in a portable telephone 10 shownin FIG. 1.

FIG. 3 schematically shows setting of a focus area and extraction of animage in a focus area extraction portion 21 shown in FIG. 2.

FIG. 4 shows an example of spatial differentiation operators used foredge enhancement processing of image signals in an edge enhancementprocessing portion 22 shown in FIG. 2.

FIG. 5 shows a form where a focus state is displayed on LCD displayportions 115 and 116 of a portable telephone 10 according to a firstembodiment of the present invention where the value of the focus stateis displayed using a bar graph.

FIG. 6 shows a form where a focus state is displayed on LCD displayportions 115 and 116 of a portable telephone 10 according to a firstembodiment of the present invention where the value of the focus stateis displayed using a level meter.

FIG. 7 shows a form where a focus state is displayed on LCD displayportions 115 and 116 of a portable telephone 10 according to a firstembodiment of the present invention where the value of the focus stateis displayed using numerical values.

FIG. 8 shows a form where a focus state is displayed on LCD displayportions 115 and 116 of a portable telephone 10 according to a firstembodiment of the present invention where the value of the focus stateis displayed using a predetermined number of graphic forms.

FIG. 9 shows a form where a focus state is displayed on LCD displayportions 115 and 116 of a portable telephone 10 according to a firstembodiment of the present invention where the value of the focus stateis displayed using a color-displayed block.

FIG. 10 shows an example of the constitution of a color table forrecording the relationship between the colors of blocks 91 and the valueof a focus state in a display form shown in FIG. 9.

FIG. 11 shows a form where a focus state is displayed on LCD displayportions 115 and 116 of a portable telephone 10 according to a firstembodiment of the present invention where the value of the focus stateis displayed using the density of displayed characters.

FIG. 12 shows a functional block diagram indicating the constitution ofa focus state display apparatus disposed in a camera-equipped portabletelephone as a fourth embodiment of the present invention.

FIG. 13 shows a graph indicating a histogram generated concerning therelationship between edge intensity and frequency relative to the valuesof image data signals subjected to edge enhancement in a histogramprocessing portion 23 of FIG. 2.

FIG. 14 shows a functional block diagram indicating an example of aschematic constitution of a portable telephone according to theembodiments of the present invention.

FIG. 15 schematically shows an example of a constitution of a lens focaldepth switching mechanism of a camera 112, indicating an example of afocal depth switching mechanism for a normal photographing and an OCRfunction mode.

FIG. 16 shows a flow chart diagram describing, with the focus on an OCRfunction, the flow of processing in a camera-equipped portable telephonecomprising a focus state display apparatus according to a fifthembodiment of the present invention.

FIG. 17 shows a flow chart diagram describing, with the focus on an OCRfunction, the flow of processing in a camera-equipped portable telephonecomprising a focus state display apparatus according to a fifthembodiment of the present invention.

FIG. 18 shows a flow chart diagram describing, with the focus on an OCRfunction, the flow of processing in a camera-equipped portable telephonecomprising a focus state display apparatus according to a fifthembodiment of the present invention.

FIG. 19 shows a flow chart diagram describing, with the focus on an OCRfunction, the flow of processing in a camera-equipped portable telephonecomprising a focus state display apparatus according to a fifthembodiment of the present invention.

FIG. 20 shows a flow chart diagram describing, with the focus on an OCRfunction, the flow of processing in a camera-equipped portable telephonecomprising a focus state display apparatus according to a fifthembodiment of the present invention.

FIG. 21 shows contents displayed on display means in the flow ofprocessing in a camera-equipped portable telephone comprising a focusstate display apparatus according to a fifth embodiment of the presentinvention.

FIG. 22 shows contents displayed on display means in the flow ofprocessing in a camera-equipped portable telephone comprising a focusstate display apparatus according to a fifth embodiment of the presentinvention.

FIG. 23 shows contents displayed on display means in the flow ofprocessing in a camera-equipped portable telephone comprising a focusstate display apparatus according to a fifth embodiment of the presentinvention.

FIG. 24 shows contents displayed on display means in the flow ofprocessing in a camera-equipped portable telephone comprising a focusstate display apparatus according to a fifth embodiment of the presentinvention.

FIG. 25 shows contents displayed on display means in the flow ofprocessing in a camera-equipped portable telephone comprising a focusstate display apparatus according to a fifth embodiment of the presentinvention.

FIG. 26 shows contents displayed on display means in the flow ofprocessing in a camera-equipped portable telephone comprising a focusstate display apparatus according to a fifth embodiment of the presentinvention.

FIG. 27 shows contents displayed on display means in the flow ofprocessing in a camera-equipped portable telephone comprising a focusstate display apparatus according to a fifth embodiment of the presentinvention.

FIG. 28 shows contents displayed on display means in the flow ofprocessing in a camera-equipped portable telephone comprising a focusstate display apparatus according to a fifth embodiment of the presentinvention.

FIG. 29 shows contents displayed on display means in the flow ofprocessing in a camera-equipped portable telephone comprising a focusstate display apparatus according to a fifth embodiment of the presentinvention.

FIG. 30 shows contents displayed on display means in the flow ofprocessing in a camera-equipped portable telephone comprising a focusstate display apparatus according to a fifth embodiment of the presentinvention.

FIG. 31 shows contents displayed on display means in the flow ofprocessing in a camera-equipped portable telephone comprising a focusstate display apparatus according to a fifth embodiment of the presentinvention.

FIG. 32 illustratively shows a manner where a URL printed on a namecard, for example, is photographed using a camera-equipped portabletelephone according to the present embodiment.

BEST MODE FOR CARRYING-OUT OF THE INVENTION First Embodiment

A first embodiment of the present invention is a camera-equippedportable telephone/PHS (Personal Handy-Phone System) comprising a focusstate display apparatus (hereafter simply abbreviated as “portabletelephone” so as to simplify the description). FIG. 1 shows an outwardappearance of a camera-equipped portable telephone comprising a focusstate display apparatus. FIG. 1(A) shows a rear perspective view of theportable telephone of the present embodiment, and FIG. 1(B) shows afront perspective view thereof. FIG. 14 shows a functional block diagramindicating an example of a schematic constitution of the portabletelephone according to the present embodiment.

In FIG. 1, a camera-equipped portable telephone 10 comprises a bodyportion 10 a and an antenna 111 for performing radio communication. Theback face of the body portion 10 a comprises a camera 112 (imagingmeans) for photographing an image, an LED (light emitting diode) 113(light emitting means) for notifying a focus state by blinking orlighting, a loudspeaker 114 (sound-producing means) for notifying thefocus state using a sound or a voice, and a small LCD (liquid crystaldisplay) display portion 116 (display means) for displaying a subscreen.

Also, the front face of the body portion 10 a comprises an LCD displayportion 115 (display means) for displaying a main screen, a power supplykey 117 for turning on/off a power supply, a telephone function key 118for operating incoming calls, outgoing calls, switching of telephonefunctions, determination of performance, and the like, a mode key 119for switching various functions, cursor keys 120 for moving selectionobjects in the up and down and right and left directions, a shatter key121 for determining the photographing of an image to be inputted fromthe camera 112, dial keys 122 including membrane keys for dialing suchas inputting telephone numbers, a loudspeaker 123 for voiceamplification to perform call reception, and a microphone 124 forinputting a voice.

In this case, the camera 112 is a small and light camera to be disposedin portable devices. For example, the camera may be a CCD (ChargeCoupled Device, an area-type solid-state imaging device) camera of 300thousand pixels.

Moreover, LCD display portions 115 and 116 are constituted using devicessuch as an LCD display having a dot matrix structure, a PDP (PlasmaDisplay Panel) as a back light, and an organic EL (ElectronLuminescence), and various drivers as an interface between these devicesand the inner system of the portable telephone 10.

When the portable telephone 10 is operated, guidance, receptioninformation, images, text information, icons, and the like to performvarious operations are displayed on the LCD display portions 115 and116.

When photographing is performed using the camera 112, the focus state isdisplayed on the LCD display portions 115 and 116 along with an imageinputted from the camera 112. When the user adjusts the position of thecamera 112, the focus state of a photographing target varying inaccordance therewith can be recognized. When the focus is adjusted,information thereof is displayed on the LCD display portions 115 and 116and notified to the user by generating a predetermined sound via theloudspeaker 114 or the loudspeaker 123 for voice amplification.

In the portable telephone of the present embodiment, the aforementioneddisplay of the focus state and the notification that the focus isadjusted can be performed by any one of, or a combination of the LCDdisplay portions 115 and 116, the LED 113, and the loudspeaker 114 orthe loudspeaker 123 for voice amplification.

As shown in FIG. 14, the portable telephone 10 according to the presentembodiment comprises a CPU (control portion) 231 for controlling theentire apparatus (including photographing procedure switching means,recognition means (OCR function) for characters, codes, and the like,information processing means, frame rate control means, timer settingmeans, and the like), a memory 232 including semiconductor memories suchas a ROM for storing a control program, fixed data, and the like, and aRAM as a storage area for the operation of the CPU, and a hard diskdrive, the memory storing received or inputted character information,image information, and voice signals. The portable telephone 10 furthercomprises the camera 112 (imaging means) for inputting image informationhandled as a target, a light 234 (lighting means) for emitting light byuser operation upon imaging, for example, the LCD display portion 115(display means, a portion of output (notification) means) for displayingcharacter information and image information to the user, the key inputportion 122 for inputting and generating key operation information fromthe user, the antenna 111 for transmitting and receiving radio waves, aradio portion 238 for controlling the transmission and reception ofradio communication, the loudspeaker 114 (a portion of output(notification) means) for outputting a voice signal, and the microphone124 for inputting a voice signal from the user.

The control portion 231 controls the entire portable telephone 10 andalso has an OCR function. The control portion photographs characters,for example, using the camera 112, displays image information,recognizes characters, for example, and performs processing inaccordance with recognized contents on the basis of key operation.

Especially, the control portion 231 has photographing procedureswitching means for switching at least a portion of processing betweenwhen photographing natural images, for example, and when photographingin the OCR function mode, where characters, a bar code, and the like arephotographed for the purpose of character recognition (OCR), capturing,or the like. The control portion 231 further has the OCR function forrecognizing characters, a bar code, and the like when the processing isswitched to the OCR function mode, notification means for notifying theuser of results recognized via the OCR function, for example, and afunction as information processing means for performing a predeterminedprocessing on the basis of contents recognized via the OCR function.

Moreover, the control portion 231 has a function as frame rate controlmeans by which when the processing is switched to the OCR function mode,the control portion displays an image on the display portion 115 at afast frame rate that can be obtained from the camera 112 until the OCRfunction is started, and after the OCR function is started, the image isphotographed and displayed at a frame rate that is necessary for the OCRfunction.

The ROM is a read-only semiconductor memory for storing fixed data suchas a program and communication control data necessary when the controlportion 231 operates. The RAM is used as what is called a working memoryfor temporarily storing data concerning bar code recognition andcommunication, data used for calculation, calculation results, and thelike. A program to be processed in the portable telephone 10 is expandedin the RAM, and then executed.

FIG. 15 schematically shows an example of the constitution of a lensfocal depth switching mechanism of the camera 112, indicating an exampleof a focal depth switching mechanism for a normal photographing and theOCR function mode. As shown in FIG. 15, the lens focal depth switchingmechanism comprises a camera module 250 having a solid-state imagingdevice such as a CCD and a signal output circuit thereof, a lens unit252 fitted on the camera module 250 with a screw, for example, the lensunit 252 having a lens 251 in the inside thereof, and a camera lever 253installed on the peripheral portion of the lens unit 252, the cameralever 253 turning the lens unit 252. The lens unit 252 is fitted on thecamera module 250 using a screw. When a knob 253 a of the camera lever253 is moved in the A←→B direction, the lens unit 252 is moved in theX←→Y direction relative to the camera module 250. Thus, the focaldistance between the lens 251 attached to the lens unit 252 and thecamera module 250 is switched, so that the focal depth between thenormal photographing and when the OCR function is in operation isswitched. In the present embodiment, when the OCR function is inoperation, the camera lever 253 slides by a driving portion (omitted inthe drawings) including a solenoid that has received a driving signalfrom the control portion 231 (FIG. 14), for example, and then the focaldepth is automatically switched. A sliding knob interlocking with theknob 253 a of the camera lever 253 may be disposed on a camera body tohave constitution where the switching is performed by manual operationof the user upon macro photographing when the OCR function is inoperation, for example.

FIG. 2 shows a functional block diagram indicating the constitution ofthe focus state display apparatus disposed in the portable telephone 10of the present embodiment.

In FIG. 2, the focus state display apparatus of the present embodimentcomprises an imaging device 20 (corresponding to the camera 112 of theportable telephone 10), a focus area extraction portion 21, an edgeenhancement processing portion 22, a histogram processing portion 23, avariance value calculation processing portion 24, a time integrationvalue calculation processing portion 25, focus state determination means26, focus state display means 31, and a display image synthesis portion32.

An image captured from the imaging device 20 is processed in the focusarea extraction portion 21, the edge enhancement processing portion 22,the histogram processing portion 23, the variance value calculationprocessing portion 24, the time integration value calculation processingportion 25, the focus state determination means 26, and the focus statedisplay means 31, thereby generating a display image of the focus state.The display image of the focus state and the image captured from theimaging device 20 are synthesized in the display image synthesis portion32, and then outputted to the LCD display portions 115 and 116.

In the focus state display apparatus of the present embodiment and thecamera-equipped portable telephone provided therewith, components otherthan described above are constituted in the same manner as conventionalones, and description thereof is omitted in this case.

Next, the operations of the portable telephone 10 and the focus statedisplay apparatus in the present embodiment above are described withreference to FIG. 2.

When an image is photographed using the camera 112 of the portabletelephone 10 of the present embodiment, the image captured in theimaging device 20 is provided with a focus area setting in the focusarea extraction portion 21, and then luminance signals of image signalsincluded in the range of the focus area are extracted.

FIG. 3 schematically shows the setting of the focus area and theextraction of the image in this case. In FIG. 3, a rectangular focusarea 42 is set in the central portion of an image 41 captured via theimaging device 20, and image signals included in the focus area 42 areextracted. The size, shape, position and the like of the focus area canbe changed arbitrarily and may also be set arbitrarily by the user.

The luminance signals extracted in the focus area extraction portion 21are subjected to local multiply and accumulate processing, and thensubjected to absolute value processing in the edge enhancementprocessing portion 22 using spatial differentiation operators, therebygenerating edge-enhanced image signal data. FIG. 4 shows an example ofthe spatial differentiation operators used in the edge enhancementprocessing portion 22.

After the procedure, the focus state display apparatus of the presentembodiment has the following two methods to determine the focus state onthe basis of the aforementioned edge-enhanced image signal data.

In the first method, in the time integration value calculationprocessing portion 25, a time integration value in which edge-enhancedsignals in a certain period of time are integrated is calculated. Inthis case, a certain period of time is set as time necessary to reducean influence resulting from a temporal fluctuation of the values of thesignals above and to average the image signals. This stabilizes thedetermination of the focus state. Next, in the focus state determinationmeans 26, the value of the focus state is calculated by dividing thetime integration value calculated in the value calculation processingportion 25 by a predetermined value, and then by adding an offset value.If the value of the focus state is large, the focus is determined asadjusted, and if the value of the focus state is small, the focus isdetermined as unadjusted.

In the second method, in the histogram processing portion 23, ahistogram as shown in FIG. 13 is generated concerning the relationshipbetween edge intensity and frequency relative to the values of theedge-enhanced signals. Then, in the variance value calculationprocessing portion 24, a variance value of signals is calculated usingonly those signals not less than a certain threshold value in theaforementioned histogram. Next, in the time integration valuecalculation processing portion 25, a time integration value iscalculated in which the calculated variance value above is integrated ina certain period of time. In this case, a certain period of time is timenecessary to reduce an influence resulting from a temporal fluctuationof the values of the image signals and to average the image signals inthe same manner as mentioned above. However, time longer than a certainperiod time in the first method is set.

Next, in the focus state determination means 26, the value of the focusstate is calculated by dividing the time integration value of the imagesignals calculated in the value calculation processing portion 25 by apredetermined value, and then by adding an offset value. If the value ofthe focus state is large, the focus is determined as adjusted, and ifthe value of the focus state is small, the focus is determined asunadjusted.

In the aforementioned second method, only those signals not less than acertain threshold value in the histogram are used in order to removeedges generated due to high frequency noises in the focus area 42, handmovement of the user, or the like and to control the influence thereof.

Also, when calculating the variance value from the histogram, a methodfor obtaining the variance value may be employed by which only thosevalues not less than a certain threshold value and values not more thana certain threshold value are used. This removes edges detected based onrandom noises generated when photographing in the dark.

In general, when the focus of a manual camera is adjusted, the focus isadjusted by moving a focus ring. However, in a case involving afixed-focus camera such as in a portable terminal device, the user isrequired to move up and down the portable terminal in order to adjust toa predetermined focal distance. Thus, in the imaging device 20 of thepresent embodiment, the image in the focus area 42 shown in FIG. 3changes greatly in the up-down direction, and the time integration valueof image signals is greatly fluctuated.

As mentioned above, by calculating the time integration value using thevariance value calculated in the variance value calculation processingportion 24, an influence resulting from the change of the image in thefocus area can be controlled, and a stable value can be calculated.

When the aforementioned first method is compared with the second method,although the first method has lower accuracy in the value of thecalculated focus state than the second method, the first method isadvantageous in that it takes less time for processing. By contrast,although the second method takes more time for processing than the firstmethod, the second method is advantageous in that it has higher accuracyin the value of the calculated focus state.

In the present embodiment, either the first method or the second methodmay be used. Or, the time integration value may be obtained by bothmethods such that a value of better (higher) accuracy is employed.

Also, in the present embodiment, in the aforementioned first method andthe second method, the focus area extraction portion 21 extracts theluminance signals of image signals included in the range of the focusarea from an image captured in the imaging device 20. This reduces timerequired for the following calculation process and prevents thedeterioration of accuracy in the determination of the focus stateresulting from the influence of unnecessary image portions. However, thepresent invention is not limited to this, and in a case where a CPUcapable of high-speed processing is used or there are no unnecessaryimage portions, the luminance signals of image signals may be extractedregarding all images captured in the imaging device 20 withoutperforming extraction via the focus area extraction portion 21.

Also, in the present embodiment, in the aforementioned first method andthe second method, by performing edge enhancement via the edgeenhancement processing portion 22, accuracy in the determination of thefocus state in an image having unclear edge portions is improved.However, the present invention is not limited to this. If an imagehaving unclear edge portions does not exist, the luminance signalsextracted via the focus area extraction portion 21 or the luminancesignals of the image signals regarding all images captured in theimaging device 20 may be used in the following processing withoutperforming the edge enhancement via the edge enhancement processingportion 22.

Moreover, in the present embodiment, in the aforementioned secondmethod, the time integration value calculation processing portion 25calculates a time integration value by integrating a variance value in acertain period of time, which is calculated via the variance valuecalculation processing portion 24. This reduces an influence resultingfrom a temporal fluctuation of the values of signals and improves theaccuracy in the determination of the focus state. However, the presentinvention is not limited to this. The variance value calculated via thevariance value calculation processing portion 24 may be used in thefollowing process.

In the above, the value of the focus state calculated via the focusstate determination means 26 is inputted into the focus state displaymeans 31. The focus state display means 31 performs processing fordisplaying and notifying the focus state to the user on the basis of thevalue of the focus.

The focus state display means 31 generates a focus state display imagecorresponding to the value of the focus state, and outputs the image tothe display image synthesis portion 32. Also, the focus state displaymeans 31 can output a signal for lighting or blinking the LEDblinking/lighting device 113 or a signal for causing the loudspeakers114 and 123 to produce a sound in a predetermined occasion.

In this case, the aforementioned predetermined occasion is a case wherethe value of the focus state becomes the maximum value, for example, andin this occasion, the focus is adjusted in the imaging device 20. It ispossible to notify the user of this information by lighting or blinkingthe LED blinking/lighting device 113 or causing the loudspeakers 114 and123 to produce a predetermined sound.

In the following, an example of display of the focus state on the LCDdisplay portions 115 and 116 is specifically described with reference toFIGS. 5 to 11. The focus state display means 31 is set in advance suchthat it can perform these displays.

FIGS. 5 to 11 show illustrations of forms to display the focus state onthe LCD display portions 115 and 116 of the portable telephone 10 of thepresent embodiment, comprising the focus state display apparatus asmentioned above. In a display form shown in FIG. 5, the focus state isdisplayed using a bar graph. In FIG. 5, a focus state display graph 51is displayed on a display screen 53 on the LCD display portion 115 or116, indicating that as a gauge moves to the right, the value of thefocus state becomes higher. Also, a bar 52 indicates the maximum valueof the focus state so far.

On the basis of such a focus state display, the user can performphotographing, moving up and down the portable telephone 10 of thepresent embodiment while viewing the display screen 53 and determining aposition where the gauge of the focus state display graph 51 becomes themaximum as the focus position of the camera 112.

In a display form shown in FIG. 6, the focus state is displayed using alevel meter. In FIG. 6, a focus state displaying level meter 61 isdisplayed on a display screen 63 on the LCD display portion 115 or 116,indicating that as the display level of the level meter increasesclockwise from min to max, the value of the focus state becomes higher.Also, a bar 62 indicates the maximum value of the focus state so far.

On the basis of such a focus state display, the user can performphotographing, moving up and down the portable telephone 10 of thepresent embodiment while viewing the display screen 63 and determining aposition where the display level of the focus state displaying levelmeter 61 becomes the maximum as the focus position of the camera 112.

In a display form shown in FIG. 7, the focus state is displayed usingnumerical values. In FIG. 7, on a display screen 73 on the LCD displayportion 115 or 116, a numerical value corresponding to the focus stateis displayed on a display window 71, indicating that as the value of thenumerical value becomes higher, the value of the focus state becomeshigher. Also, a display window 72 displays the maximum value of thefocus state so far.

On the basis of such a focus state display, the user can performphotographing, moving up and down the portable telephone 10 of thepresent embodiment while viewing the display screen 73 and determining aposition where the numerical value corresponding to the focus statedisplayed on the display window 71 becomes the maximum as the focusposition of the camera 112.

In a display form shown in FIG. 8, the focus state is displayed usingthe number of graphic forms. In FIG. 8, graphic forms 81 whose numbercorresponds to the focus state are displayed on a display screen 83 ofthe LCD display portion 115 or 116, indicating that as the number of thegraphic forms 81 becomes greater, the value of the focus state becomeshigher. In order to perform such a display, graphic forms correspondingto a quotient obtained by dividing the value of the focus state by apredetermined value may be displayed, for example. Also, a graphic form82 displays the number of graphic forms that corresponds to the maximumvalue of the focus state so far.

Although rectangular quadrangles are displayed as an example in FIG. 8,other than this shape, any shape of graphic form may be used fordisplay.

On the basis of such a focus state display, the user can performphotographing, moving up and down the portable telephone 10 of thepresent embodiment while viewing the display screen 83 and determining aposition where the number of displayed graphic forms 81 becomes themaximum as the focus position of the camera 112.

In a display form shown in FIG. 9, the focus state is displayed usingthe variation of color. In FIG. 9, a block 91 whose color corresponds tothe focus state is displayed on a display screen 93 on the LCD displayportion 115 or 116. The color of this block 91 corresponds to the valueof the focus state.

FIG. 10 shows an example of the constitution of a color table forrecording the relationship between the colors of the blocks 91 and thevalue of the focus state in this case. Normally, the colors of theblocks 91 displayed on the display screen 93 are determined using RGBcolor elements. The values of these color elements may be associatedwith the value of the focus state and recorded in the color table.

The example shown in FIG. 10 is constituted such that as the color ofthe blocks 91 comes close to green, the value of the focus state becomeshigher and that as the color of the blocks 91 comes close to red, thevalue of the focus state becomes lower. As a matter of course, displayform by the focus state display apparatus of the present embodiment isnot limited to this, so that color tables of any constitution can beused.

On the basis of such a focus state display, the user can performphotographing, moving up and down the portable telephone 10 of thepresent embodiment while viewing the display screen 93 and determining aposition where the displayed blocks 91 come closest to green as thefocus position of the camera 112.

In a display form shown in FIG. 11, the focus state is displayed usingthe density of displayed characters. In FIG. 11, the characters 101“Focus is adjusted” are displayed on a display screen 103 on the LCDdisplay portion 115 or 116 in density corresponding to the value of thefocus state.

In this case, the density of the characters 101 is constituted such thatit fluctuates within a predetermined range depending on the value of thefocus state. For example, the density can be displayed in a range from 0(black) to 255 (white). On the basis of such a focus state display, theuser can perform photographing, moving up and down the portabletelephone 10 of the present embodiment while viewing the display screen103 and determining a position where the displayed characters 101 becomedarkest as the focus position of the camera 112.

Second Embodiment

In the present embodiment, since the constitution of the focus statedisplay apparatus and the portable telephone 10 provided therewith isthe same as in the first embodiment shown in FIGS. 1 and 2, descriptionthereof is omitted. The first embodiment is constituted such that thefocus state is displayed on the display screens of the LCD displayportions 115 and 116 as shown in FIGS. 5 to 11. By contrast, the secondembodiment is characterized in that, instead of or in addition to thedisplay on the LCD display portions 115 and 116, the focus state isnotified to the user by blinking or lighting the LED 113 disposed in theportable telephone 10 by a predetermined method.

As a manner for blinking the aforementioned LED 113 by a predeterminedmethod, for example, in FIG. 2, the focus state display means 31 can beconstituted such that it calculates a value obtained by dividing apredetermined value by the value of the focus state at all times andblinks the LED 113 using the calculated value as a pace. This increasesthe pace of the blinking of the LED 113 in proportion as the value ofthe focus state becomes higher, and the user can determine the focusstate of the camera 112 in accordance therewith.

Also, as a manner for blinking the aforementioned LED 113 by apredetermined method, for example, in FIG. 2, the focus state displaymeans 31 can be constituted such that it determines whether the value ofthe focus state is greater than a predetermined value, and causes theLED 113 to light if the value of the focus state is greater than thepredetermined value. This causes the LED 113 to light when the value ofthe focus state exceeds the predetermined value, so that the user candetermine the focus state of the camera 112 in accordance therewith.

Third Embodiment

In the present embodiment, since the constitution of the focus statedisplay apparatus and the portable telephone 10 provided therewith isthe same as in the first embodiment shown in FIGS. 1 and 2, descriptionthereof is omitted. The first embodiment is constituted such that thefocus state is displayed on display screens, namely, the LCD displayportions 115 and 116 as shown in FIGS. 5 to 11. By contrast, the secondembodiment is characterized in that, instead of or in addition to thedisplay on the LCD display portions 115 and 116, the focus state isnotified to the user by causing the loudspeaker 114 or 123 disposed inthe portable telephone 10 to produce a predetermined sound.

As a manner for causing the aforementioned loudspeaker 114 or 123 toproduce the predetermined sound, for example, in FIG. 2, the focus statedisplay means 31 can be constituted such that it calculates a valueobtained by dividing a predetermined value by the value of the focusstate at all times and causes the loudspeaker 114 or 123 to produce asound whose frequency corresponds to the calculated value. This raisesthe frequency of the sound emitted via the loudspeaker 114 or 123 inproportion as the value of the focus state becomes higher, and the usercan determine the focus state of the camera 112 in accordance therewith.

Also, as a manner for causing the aforementioned loudspeaker 114 or 123to produce the predetermined sound, for example, in FIG. 2, the focusstate display means 31 can be constituted such that it determineswhether the value of the focus state is greater than a predeterminedvalue, and causes the loudspeaker 114 or 123 to produce thepredetermined sound if the value of the focus state is greater than thepredetermined value. This causes the loudspeaker 114 or 123 to producethe predetermined sound when the value of the focus state exceeds thepredetermined value, so that the user can determine the focus state ofthe camera 112 in accordance therewith.

Fourth Embodiment

A fourth embodiment of the present invention is a camera-equippedportable telephone comprising a focus state display apparatus in thesame manner as in the first embodiment. FIG. 12 shows a functional blockdiagram indicating the constitution of the focus state display apparatusdisposed in the portable telephone. In the figure, components providedwith the same reference in FIG. 2 indicate the same element anddescription thereof is omitted.

In FIG. 12, the focus state display apparatus of the present embodimentcomprises a BPF (band-pass filter) 80 (filter means) for removing thehigh-frequency components of the spatial frequency components of imagedata signals instead of the display image synthesis portion 32 shown inFIG. 2.

The BPF 80 applies orthogonal transformation such as FFT (fast Fouriertransform), DCT (discrete cosine transform), and Hadamard transform toimage data signals outputted from the imaging device 20 in each of 8×8and 16×16 pixel blocks, for example. After inverse transformation usingonly those frequency components in a predetermined range of each bock,the image data signals can be outputted to the LCD display portions 115and 116.

The focus state display means 31 controls the BPF 80 such that itexpands the range (high-frequency components) of frequency components tobe removed from the image data signals in the BPF 80 in proportion asthe value of the focus state outputted from the focus statedetermination means 26 becomes low. In the above, a manner where therange of frequency components to be removed from the image data signalsvia the BPF 80 depending on the value of the focus state may be asfollows, for example.

The focus state display means 31 calculates a value obtained by dividingthe value of the focus state by a predetermined value. After the BPF 80orthogonally transforms the image data signals in each pixel block, theBPF 80 obtains the aforementioned value calculated via the focus statedisplay means 31. Then, from the spatial frequency components includedin the image data signals, the BPF 80 extracts lower frequencycomponents successively so that the ratio of the number of the extractedfrequency ones to the total is equal to the ratio value (percentage, forexample) corresponding to the aforementioned value, transforms inverselythe extracted frequency ones with other frequency components set to 0,and outputs them to the LCD display portions 115 and 116.

On the basis of such a focus state display, the user can performphotographing, moving up and down the portable telephone 10 of thepresent embodiment while viewing the display screens of the LCD displayportions 115 and 116 and intuitively determining a position where a blurof a displayed image is small as the focus position of the camera 112.

According to the camera-equipped portable telephone and the focus statedisplay apparatus disposed therein in each of the embodiments above,after image data signals captured from the camera 112 are subjected to apredetermined processing in the focus area extraction portion 21, theedge enhancement processing portion 22, the histogram processing portion23, the variance value calculation processing portion 24, the timeintegration value calculation processing portion 25, and the like, thefocus state thereof is determined via the focus state determinationmeans 26. And the focus state can be displayed or notified in anintuitive method that is readily understandable to the user via thefocus state display means 31 through the LCD display portions 115 and116, the LED 113, and the loudspeakers 114 and 123.

This allows the user who intends to photograph an image using theportable telephone 10 to readily confirm and adjust the focus byreferring to the focus state displayed in a visual or acoustic manner.

Fifth Embodiment

In the present embodiment characters are included in the photographingtargets. The “characters” in this case include codes such as bar codesin addition to general languages and characters. In other words, thecharacters mean display in which the photographing targets per se arearranged to have a general or universal meaning.

Nowadays, the improvement of the functions of camera-equipped portabletelephones is remarkable. For example, the development of portabletelephones having a number of pixels at 100 thousand pixel level is inprogress. Also, with the improvement of the number of pixels, variousapplications that have been difficult in the past have become possible.One of them is thought to be mounting an OCR function on portabletelephones. However, the number of pixels of conventional portabletelephones is about 100 thousands, so that it has been difficult todirectly photograph items printed on name cards, for example, namelycharacters such as names, company names, telephone/fax numbers, mailaddresses, and the like using a camera disposed in the portabletelephones and to obtain the characters in the portable telephones.

The portable telephone of the present embodiment employs the focus statedisplay apparatus according to each of the aforementioned embodiments.In the camera-equipped portable telephone, by improving the number ofpixels of a digital camera to a million pixel level, the number ofpixels is equivalent to that of a small digital still camera and imaginghaving a picture quality satisfactory to the user becomes possible. Theinventors thought that by disposing an OCR function in such acamera-equipped portable telephone that is highly pixelated, althoughconventional portable telephones have difficulty, it becomes possible todirectly photograph characters such as names, company names,telephone/fax numbers, mail addresses, and the like using a cameradisposed in the portable telephone, obtain them in the portabletelephone, thereby storing in a memory, and to directly make a phonecall or send a mail on the basis of the obtained telephone numbers andmail addresses. In this occasion, if the number of pixels becomes large,it becomes possible to photograph long character strings (such as URLs)with a distant focus distance.

However, when trying to recognize a long character string at a time,characters per se become small, so that a camera is preferably capableof changing from a general photographing mode to a close-up mode withease. However, in conventional apparatuses, technical consideration isnot given in this respect. Also, in order to photograph a smallcharacter string in a good state and recognize it accurately,preferably, the character string as a photographing target uponphotographing is accurately positioned on the display screen. However,in conventional apparatuses, technical consideration is not given inthis respect either.

FIG. 32 illustratively shows a manner where a URL printed on a namecard, for example, is photographed using the camera-equipped portabletelephone according to the present embodiment. As shown in FIG. 32, whenthe angle of view of a lens LS is 40° and a distance H where the focusis adjusted in a close-up state is 10 cm, the width W of a photographicsubject capable of photographing is 7.28 cm. In this width, a URLprinted on a name card NC, for example, can be photographed at a time.However, a character string displayed on display means (both a main LCDand a sub LCD) becomes small, so that it becomes very difficult toconfirm whether the focus is adjusted or not, where the focus statedisplay means according to each of the embodiments above performseffectively.

In the following, the camera-equipped portable telephone comprising thefocus state display apparatus according to the fifth embodiment of thepresent invention is described with reference to the drawings. FIGS. 16to 20 show flow chart diagrams describing, with the focus on the OCRfunction, the flow of processing in the camera-equipped portabletelephone comprising the focus state display apparatus according to thepresent embodiment. FIGS. 21 to 31 show contents displayed on thedisplay means in the flow of processing.

In a standby screen 301, when the device is set to a camera mode and acamera starting process is performed in step SI, the phrase “Camera isstarting” is displayed on a display screen 302 (step S2). In step S3,when the camera start is completed, a through image is displayed on thedisplay means (FIG. 21, 303). On this screen, “Press shutter”, “Displaymenu”, and “Cancel” can be selected. In step S4, a photographing imageis saved in a first memory by overwriting. In this case, the firstmemory is what is called a buffer memory and the first memorytemporarily saves image signals as an imaging target by constantlyoverwriting such image signals in the photographing operation. In stepS5, the photographing image saved in the first memory is displayed ondisplay means (309). When canceling the photographing, by selecting“Cancel”, the process returns to a standby screen (standby mode) (310)in step S15. In this case, when performing a normal photographing inpractice, step S7 is omitted (No in pressing a menu button), and apressing operation is performed on a shutter button (Yes) in step S8.

Processing from steps S4 to S8 is repeated until the shutter button ispressed.

When the shutter button is pressed in step S8, a confirmation screen isdisplayed in step S9 (FIG. 21, 304). In this case, when “Cancel” isselected (step S10), the process returns to step S4. When “Save” isselected in step S11 instead of selecting “Cancel”, “Saving” isdisplayed (306 of FIG. 21, 311 to 312 of FIG. 22). When “Cancel” isselected within a predetermined time during saving (No in step S11), theprocess returns to step S9. When the predetermined time has elapsedafter selecting “Save”, image data of the entire photographing image canbe practically saved. In other words, in step S12, the photographingimage saved in the first memory is compressed in JPEG format, forexample, and in step S13, the compressed image data is saved in a secondmemory (such as a non-volatile memory card) (step S13, 313 to 314 ofFIG. 22.) Then, when the process goes to step S14 to suspend the camerafunction, the process enters the standby mode in step S15 and displaythereof is performed (step S15, 301 of FIG. 21, 310 of FIG. 22).

In step S7, when pressing the menu button is selected (Yes), the processgoes to step S16 and a photographing submenu screen is displayed (FIG.21, 305). On the submenu, for example, items “1” to “9” relating tophotographing are displayed. In step S18, a desired item is selectedfrom the displayed submenu, and in step S18, whether to switchphotographing modes or not is determined. In a case where the switchingof the photographing modes is not performed, the selected item is set.In a case where the switching of the photographing modes is performed(Yes), the process goes to step S19 and a photographing mode menu screenis displayed (FIG. 21, 307). In the photographing mode menu, items “1”to “4” are displayed, for example. Depending on which item is selectedto perform a determination process, the process can go to any one ofsteps S21 to S24 (A to D of FIG. 16). When “Still image” in step S21 isselected, and then “Yes” is selected. the process returns to step S4 toperform processing.

When “Motion picture” in step S22 is selected, as shown in FIG. 17, theprocess goes to step S25 and a photographing image is saved in the firstmemory by overwriting. Next, in step S26, the photographing image savedin the first memory is displayed, and in step S27, whether to press thecancellation button or not is determined. If the cancellation button ispressed (Yes), the process returns to the standby mode in step S15. In acase where the cancellation button is not pressed (No), the process goesto step S28 and whether to press the menu button or not is determined.In the following steps S29 to S38, substantially the same processing asin steps S11 to S14 of FIG. 16 is performed regarding motion picturephotographing, so that description in the flow chart diagram is omitted.

In step S24, when “Character reading” is selected (Yes, 315 of FIG. 23),the process goes to step S39 of FIG. 18, and a character reading menuscreen is displayed (FIG. 23, 316). Items of character reading include“1” URL, “2” mail address, “3” phone number, and “4” alphanumericcharacters, for example. Other items may be disposed.

In step S41, whether a URL is a target item of character reading or notis determined. In case of Yes, the process goes to step S42 and “1” isstored in an area X in a memory. In case of No in step S41, whether amail address is the target item of character reading or not isdetermined. In case of Yes, the process goes to step S43 and “2” isstored in the memory area X. In case of No in step S43, the process goesto step S45 and whether a telephone number is the target item ofcharacter reading or not is determined. In case of Yes, the process goesto step S46 and “3” is stored in the memory area X. In case of No instep S45, the process goes to step S47 and whether alphanumericcharacters are the target item of character reading or not isdetermined. In case of Yes, the process goes to step S48 and “4” isstored in the memory area X. In case of No in step S47, the process goesto step S49.

In step S49, a photographing image is saved in the first memory byoverwriting and in step S50, and in step S50, the photographing imagesaved in the first memory is displayed on the display means. In thisoccasion, as shown on a display screen 317 of FIG. 23, for example,“abcdef.com”, which is a photographing target representing the URL ofthe homepage of Abcdef, is displayed as a through image, and also afocus bar is displayed. For example, the message “Photograph when thecolor of the focus bar is closest to blue” is displayed, and the usercan confirm the focus state in accordance with this display. In otherwords, from step S51, the determination of focus is performed and instep S52, the result of the focus determination is displayed. The focusdetermination and the result display thereof may be performed byprocessing described in each of the aforementioned embodiments. When thefocus is confirmed, whether to press the shutter button or not isdetermined (step S53). In case of No in step S53, whether to press themenu button or not is determined in step S54. In a case where the menubutton is pressed (Yes), the process returns to step S19 of FIG. 16 andthe photographing mode menu screen is displayed. In a case where themenu button is not pressed (No), in step S55, whether to press thecancellation button or not is determined. In case of No in step S55, theprocess returns to step S49. In case of Yes, the process returns to stepS15 and enters the standby mode.

In case of Yes in step S53, the process goes to step S56 and thephotographing image of “abcdef.com” is displayed in a rectangular area318 a as shown on a display screen 318 of FIG. 23.

Next, in step S57, whether to press the cancellation button or not isdetermined. In case of Yes, the process returns to step S49. In case ofNo, the process goes to step S58 and whether to press a recognitionbutton or not is determined. In case of No, the process returns to stepS56. In case of Yes, in step S59, a character recognition process isperformed via an OCR function, and in step S60, the result of thecharacter recognition is displayed on the display means. Then, theprocess goes to step S61 and whether re-recognition is necessary or notis determined depending on the recognition result. In case of Yes, theprocess returns to step S49. In case of No, the process goes to stepS62. In step S62, whether to press the menu button or not is determined.In case of Yes, the process goes to step S71 of FIG. 19. In case of No,the process goes to step S63 and whether to press an enter button or notis determined. In case of No, the process returns to step S60. In caseof Yes, the process goes to step S64. In step S64, whether “1” is storedin the aforementioned memory area X or not is determined. In case ofYes, the process goes to a URL input mode in step S67. In case of No,the process goes to step S65 and whether “2” is stored in theaforementioned memory area X or not is determined. In case of Yes, theprocess goes to a mail composition mode in step S68. In case of No, theprocess goes to step S66 and whether “3” is stored in the aforementionedmemory area X or not is determined. In case of Yes, the process goes toan outgoing call mode in step S69. In case of No, the process goes to anotepad registration mode in step S70.

As shown in FIG. 19, in step S71, a character recognition submenu screenis displayed (FIG. 23, 323). Any one of items is selected from “1” to“7” of the character recognition submenu in step S72, and from theaforementioned “1” to “6” in steps S73 to S78. For example, incontinuous recognition in step S73, when “1 Continuous recognition” isselected, the process goes to step S49. If this is not selected, inadditional recognition in step S74, when “2 Additional recognition” isselected, the process goes to step S39. In new registration in atelephone book in step S75, when “3 New registration in the telephonebook” is selected, the process goes to a telephone book registrationmode in step S79. If this is not selected, in additional registration ina telephone book in step S76, when “4 Additional registration in thetelephone book” is selected, the process goes to a telephone book editmode in step S80. If this is not selected, in bookmark registration instep S77, when “Bookmark registration” is selected, the process goes toa bookmark registration mode. If this is not selected, the process goesto edit selection in step S78. When “Edit” is selected, the process goesto an input edit mode in step S82. If “Edit” is not selected, theprocess goes to a mode of a selected item. Concrete processes anddisplay of the aforementioned “1” to “3” are described in the following.

As shown in FIG. 20, in step S83 continuing from a code readingselection step of FIG. 16, a code reading menu screen is displayed (FIG.27, 352). Examples of items include two types, namely, “1” aone-dimensional bar code and “2 ” a two-dimensional bar code (QR code).In step S85, whether to select reading of a one-dimensional bar code ornot is determined. In a case where the reading of a one-dimensional barcode is selected, “1” is stored in a memory Y. Thereafter, or in a casewhere the reading of a one-dimensional bar code is not selected, theprocess goes to step S87 and whether to select reading of atwo-dimensional bar code or not is determined. In a case where thereading of a two-dimensional bar code is selected, “2” is stored in thememory Y in step S88 and the process goes to step S89. In a case wherethe reading of a two-dimensional bar code is not selected, the processgoes to step S89 without going through step S88.

Next, in step S89, a photographing image is saved in the first memory byoverwriting, and in step S90, the photographing image saved in the firstmemory is displayed on the display means. In step S91, focusdetermination is performed. In step S92, whether the focus is adjustedor not is determined. In a case where the focus is unadjusted, theprocess returns to step S89. In a case where the focus is determined asadjusted, the process goes to step S93 and code recognition isperformed. Whether the code recognition via the OCR function hassucceeded or not is determined in step S94. If the code recognition hassucceeded, the result of the code recognition is displayed in step S95and the process goes to step S96. In this manner, by performing the coderecognition process in the focus adjusted state where code recognitionis possible, the processing burden of a CPU can be reduced and thedecrease of a frame rate accompanied with the code recognition processcan also be controlled to the minimum as compared with a case where thecode recognition process is always performed.

Next, in step S96, whether to press a re-recognition button or not isdetermined. In other words, if the code recognition has not succeeded,the re-recognition button is pressed (Yes) and the process returns tostep S89. If the result of the code recognition displayed on the displaymeans is successful (No), the process goes to step S97 and whether topress the enter button or not is determined. In a case where the enterbutton is not pressed, the process returns to step S95. In a case wherethe enter button is pressed, the process goes to step S98. Ifinformation for identifying a URL is included in the result of the coderecognition, the process can go to the URL input mode in step S67.

For example, in step S56, in the display screen 317 of FIG. 23, a URL isroughly positioned in [ ], and then the focus process is performed usingthe focus bar, followed by a process for pressing the shutter, therebydisplaying the photographing image of “Abcdef.com” in the rectangulararea 318 a as shown on the display screen 318. In a case where aplurality of rows exist, it is possible to select a row to be subjectedto the recognition process by a key operation, for example. Byperforming a process for deciding the recognition process, an OCRrecognition process for reading characters (URL) is performed (319).When the recognition is completed, as shown on a display screen 320, theURL “http://Abcdef.com” obtained via the aforementioned procedure isdisplayed as an item of [URL]. By pressing the enter button, it is alsopossible to initiate an Internet connection process relative to this URL(322).

In a case where a mail address is prepared, the process can go to themail composition mode (step S68) from step S99. For example, in stepS68, in a display screen 326 of FIG. 24, a mail address is roughlypositioned in [ ], and then the focus process is performed using thefocus bar, followed by the process for pressing the shutter, therebydisplaying the photographing image of “Taro.XX@abc.com” in a rectangulararea 327 a as shown on a display screen 327. In a case where a pluralityof rows exist, it is possible to select a row to be subjected to therecognition process by a key operation, for example. By performing aprocess for deciding the recognition process, the OCR recognitionprocess for reading characters (mail address) is performed (328). Whenthe recognition is completed, as shown on a display screen 329, the mailaddress obtained via the aforementioned procedure is displayed as anitem of [Mail address]. By pressing the enter button, a mail compositionscreen relative to a person who owns the mail address is displayed(330). In this case, by pressing “Transmit”, it is possible to initiatethe Internet connection process relative to this mail address and totransmit an electronic mail (331).

In a case where a telephone number is displayed, the process can go tothe outgoing call mode in step S69 from step S100. Further, the processcan go to the notepad registration mode in step S70.

For example, in step S69, in a display screen 335 of FIG. 25, atelephone number is roughly positioned in [ ], and then the focusprocess is performed using the focus bar, followed by the process forpressing the shutter, thereby displaying the photographing image of“0123456789” in a rectangular area 336 a as shown on a display screen336. In a case where a plurality of rows exist, it is possible to selecta row to be subjected to the recognition process by a key operation, forexample. By performing a process for deciding the recognition process,the OCR recognition process for reading characters (telephone number) isperformed (337). When the recognition is completed, as shown on adisplay screen 338, the telephone number obtained via the aforementionedprocedure is displayed as an item of [Telephone number]. By pressing theenter button, an outgoing call confirmation screen relative to a personwho owns the telephone number is displayed (339). In this case, bypressing “Call”, an outgoing call process is initiated (340).

A recording process of alphanumeric characters in a notepad shown inFIG. 26 can be performed via the same procedure as in the aforementionedprocess (display screens 344 to 349). Detained description is omitted.The same applies to a URL input based on a one-dimensional bar codeshown in FIG. 27 display screens 352 to 357 and a URL input based on atwo-dimensional bar code shown in FIG. 28 display screens 360 to 366.

Next, in a case where a URL is inputted (321 of FIG. 23 or 364 of FIG.28), when the URL practically comprises a further longer characterstring, it is necessary to input a continuous character string. In sucha case, in step S71, a character recognition submenu screen is displayed(FIG. 29, 367), where “1 Continuous recognition” is selected and thendecided. As shown on a display screen 368, a continuous character string“today/index.html” is photographed, and the focus is confirmed using thefocus bar, followed by the press of the shutter, thereby enabling theOCR recognition of the character string “today/index.html” as shown indisplay screen 369 (370). When the recognition is completed, as shown ona display screen 371, the character string“http://Abcdef.com/today/index.html” where the continuous characterstring is connected is displayed. After the contents is confirmed, bypressing the enter button, the aforementioned URL can be obtained andstored in the portable telephone (372). In this case, by pressing theenter button, an Internet connection relative to the aforementioned URLcan be performed (374). On the display screen 372, by displaying themenu, it is also possible to display the submenu again.

Next, after the URL is obtained, by selecting “2 Additional recognition”from the submenu display, it is possible to additionally register any ofitems “1” to “4” following the URL. For example, in a case where a mailaddress is registered from a display screen 376 in FIG. 30, by selectingand deciding “2 Mail address”, as shown on a display screen 377, a mailaddress of XX Taro on a name card is photographed. In this case also,the focus state is confirmed using the focus bar, and “Taro.XX@abc.com”is photographed as shown on a display screen 378. In this case also, itis possible to change a selection row by a key operation. When aselection for enabling the OCR recognition is made, the recognitionprocess is performed (379), and “Taro.XX@abc.com” can be obtained as amail address (380).

On a display screen 380, when “Enter” is selected, a new mailcomposition screen is displayed (381). As shown on a display screen 382,it possible to move from a mail address to a row of URL, and to connectto the URL on the Internet, for example. Also, when the menu isdisplayed (display screen 383), items “1 ” to “3”, “6 ”, and “7 ” aredisplayed as selectable items and items “4 ” and “5 ” are displayed asunselectable items. Item “4 ” is displayed as an unselectable item sinceadditional recognition is performed and item “5 ” is displayed as anunselectable item since the menu is displayed in a state where a mailaddress is selected.

Thereafter, on a display screen 384 of FIG. 31, when “2 Additionalrecognition” is selected, for example, any one of character reading from“1 ” to “4 ” can be selected as shown on a display screen 385. In thiscase, when “3 ”, namely, “Telephone number” is selected, as shown on adisplay screen 386, the telephone number of XX Taro “0123456789” isphotographed and the focus bar is displayed. After the focus state isconfirmed using the focus bar, by pressing the shutter, the telephonenumber can be photographed (387). In this case, a selection row can bechanged moving up and down a plurality of rows. Next, by performing theOCR recognition process (385), as shown on a display screen 389, thetelephone number obtained in the display position of a telephone numbercan be registered. In this case, when “Enter” is selected, as shown on adisplay screen 390, the outgoing call confirmation screen is displayed.When “Call” is selected, a process for practically making a phone callcan be performed. When “Menu” is selected, the submenu is displayed(391). When “New registration in the telephone book” is selected, thetelephone number can be registered as shown on a display screen 392. Asshown on the display screen 392, it is possible to readily prepare anaddress book from one name card, the address book being capable ofdisplaying a name, reading thereof, a telephone number, a mail address,a personal URL, and the like in a list. By using the function of “2 ”,namely, “Additional recognition”, after a URL, a mail address, atelephone number, and the like are successively recognized, it ispossible to register all recognized data in a telephone book, forexample.

As described above, according to the portable telephone of the presentembodiment, upon photographing character data printed on a name card andthe like using a camera, by notifying the user of the focus state usingfocus means, it is possible to readily learn the focus state.Thereafter, by photographing character data, for example, in a statewhere the focus is adjusted, and by recognizing the character dataphotographed in a good state using the OCR function, it is possible toreadily obtain a relatively small and long character string, such as aURL, a mail address, and the like in the portable telephone. The focusstate display apparatus and the focus state display method of thepresent invention have been described above with reference to eachembodiment used in the camera-equipped portable telephone. However, thefocus state display apparatus and the focus state display method of thepresent invention are not limited to the scope of these embodiments. Aperson skilled in the art is capable of adding various modifications andimprovements to the focus state display apparatus and the focus statedisplay method of the present invention without departing from the scopeof the invention. Examples include display using colors, display usingthe shade of color, other graph display or display using characters,pictographic characters, and the like, and notification using a sound,vibration, or the like.

In each embodiment above, the focus state display apparatus is appliedto a portable telephone, as an example. However, the apparatus may beapplied to any devices, namely, portable information terminals such asPDAs, information processing devices such as portable personalcomputers, and the like, as long as the devices comprise a(built-in/external) camera. Further image data as an imaging target maybe anything. Moreover, in each embodiment above, although namesspecified as the focus state display apparatus and the portabletelephone are used, these are used for the sake of convenience ofdescription. Thus, obviously, names such as a focusing state displayapparatus, a communication terminal device, and a focusing state displaymethod may be used.

Also, in each embodiment above, examples of display using graphic forms,symbols, characters, and the like on the LCD display portions 115 and116 and of sound production via the loudspeakers 114 and 123 aredescribed as display or notification means of the focus state. However,visual, acoustic, or other methods not involving the aforementionedexamples may be used for a notification method. Various types of devicesand each circuit portion thereof constituting the aforementionedportable telephone and focus state display apparatus, such as the types,the number, and the connection method of a memory portion and displayportions are not limited to the aforementioned embodiments, so that anywell-known structure may be employed.

The focus state display apparatus and the focus state display method ofthe present invention, and an apparatus such as a portable telephoneprovided therewith can be realized as a program for performing afunction for displaying the aforementioned focus state. Although thisprogram may be constituted such that it is embedded in each of theaforementioned devices in advance, it may be recorded in acomputer-readable recording medium separately from each of theaforementioned devices.

In the present invention, as such a recording medium, a main memory perse may be a program medium. Also, a program reader such as a CD-ROMdrive may be a program medium, which is disposed as an external memorydevice, where reading is possible by inserting a recording mediumtherein. In both cases, a stored program may be constituted such that itis executed through access via a CPU. Or, both cases use a method bywhich the program is read and the read program is downloaded into aprogram storage area that is not shown in the drawings, where theprogram is executed. A program for such downloading is stored in eachdevice in advance.

The aforementioned program medium is a recording medium constituted suchthat it is separable from a portable terminal device or an informationprocessing device. The program medium may be a magnetic disc such as amagnetic tape and a cassette tape, an optical disk such as CD-ROM,CD-R/RW, MO, MD, DVD-ROM, DVD-RAM, and DVD-RW, a card such as PC card,Compact Flash Card (registered trademark), SmartMedia (registeredtrademark), IC card, SD card (registered trademark), and Memory Stick(registered trademark), or a medium carrying a program in a fixed mannerincluding a semiconductor memory such as Mask ROM, EPROM, EEPROM, andFlash ROM.

Further, the program medium may be a medium carrying a program in afluid manner such that the program is downloaded from a communicationnetwork via communication connection means capable of connecting to anexternal communication network such as an Internet connection provideror a server terminal. In a case where the program is downloaded from acommunication network in this manner, a program for downloading may bestored in advance or installed from other recording medium. Contentsstored in the recording medium are not limited to a program, and thecontents may be data.

In this manner, if the aforementioned program is recorded in a recordingmedium that is separable from a device body, upgrading of the program,updating for adding a new display form, and the like can be flexiblymade relative to various types of devices comprising the focus statedisplay apparatus and the focus state display method of the presentinvention. Thus, it becomes possible to provide a device that is evenmore convenient to the user.

INDUSTRIAL APPLICABILITY

As described above, according to the focus state display apparatus andthe focus state display method of the present invention, focusinformation can be displayed visually or acoustically. Thus, it becomespossible for the user to readily confirm and adjust the focus.

If the focus state display apparatus having such superiorcharacteristics is applied to portable terminal devices with a smallmonitor such as portable telephones, portable information terminals, andthe like comprising an imaging device, the user can readily confirm thefocus state, so that an image with accurately adjusted focus can bephotographed.

Also, from photographing images of portable telephones and portableinformation terminals comprising an imaging device, it becomes possibleto readily recognize characters and adjust the focus of binarized imagessuch as bar codes and QR codes, thereby improving the accuracy of thecharacter recognition and the recognition of the bar codes and QR codes.

Further, by using a focus display function and an OCR function, a longcharacter string, for example, can be recognized with good accuracy.

1. A portable terminal device comprising: a photography-target settingsection operable to set a normal photography mode for photographing animage, and an OCR function mode for photographing a character or a barcode; a display unit operable to display a photographed image; afocused-state determination section operable to determine a focusedstate of a character or a bar code photographed in said OCR functionmode; a recognition section operable to recognize the photographedcharacter or bar code; and a recognized result memory section operableto store the result of recognition made by said recognition means,wherein, upon recognition by said recognition section, the result ofrecognition is displayed by said display unit, and when the displayedresult of recognition is a URL, user confirmation of said displayedresult of recognition transitions said portable terminal device directlyinto an Internet communications preparatory state having acommunications destination field that is pre-populated during saidtransition with said just-recognized URL for connection to said URL viathe Internet.
 2. The portable terminal device according to claim 1,further comprising: a photography mode switching section operable toswitch between a normal mode and a macro mode in accordance with thesetting of a photography-target by said photography-target settingmeans.
 3. The portable terminal device according to claim 1, whereinsaid focused-state determination section is activated, and thephotographed character or bar code is recognized by said recognitionsection.
 4. The portable terminal device according to claim 1, wherein astate of focus sufficient for recognition of the photographed characteror bar code is determined by said focused-state determination section,and the photographed character or bar code is recognized by saidrecognition section.
 5. The portable terminal device according to claim1, further comprising a notification section operable to provide anotice about the result of a focus state determination made by saidfocused-state determination section.
 6. The portable terminal deviceaccording to claim 1, wherein a notice about the result of determinationof a focus state made by said focused-state determination section isprovided, and the photographed character or code is recognized.
 7. Theportable terminal device according to claim 1, wherein the portableterminal device is a portable cellular phone.
 8. The portable terminaldevice according to claim 1, wherein, when the result of recognitionmade by said recognition section contains a plurality of items of dataincluding a name, telephone number, and/or mail address, the recognizeddata is registered in a telephone directory all at the same time.
 9. Theportable terminal device according to claim 1, wherein, when a firstconfirmation input is detected by the portable terminal device, saidportable terminal device initiates communications to the recognized URL.10. A portable terminal device comprising: a photography-target settingsection operable to set a normal photography mode for photographing animage, and an OCR function mode for photographing a character or a barcode; a display unit operable to display a photographed image; afocused-state determination section operable to determine a focusedstate of a character or a bar code photographed in said OCR functionmode; a recognition section operable to recognize the photographedcharacter or bar code; and a recognized result memory section operableto store the result of recognition made by said recognition means,wherein, upon recognition by said recognition section, the result ofrecognition is displayed by said display unit, and when the displayedresult of recognition is a mail address, user confirmation of saiddisplayed result of recognition transitions the portable terminal devicedirectly to a mail creation state wherein a mail destination field ofthe created mail is pre-populated during said direct transition withsaid just-recognized mail address.
 11. The portable terminal deviceaccording to claim 10, wherein, when a first confirmation input isdetected by the portable terminal device, said portable terminal deviceinitiates communications to the recognized mail address.
 12. A portableterminal device comprising: a photography-target setting sectionoperable to set a normal photography mode for photographing an image,and an OCR function mode for photographing a character or a bar code; adisplay unit operable to display a photographed image; a focused-statedetermination section operable to determine a focused state of acharacter or a bar code photographed in said OCR function mode; arecognition section operable to recognize the photographed character orbar code; and a recognized result memory section operable to store theresult of recognition made by said recognition means, wherein, uponrecognition by said recognition section, the result of recognition isdisplayed by said display unit, and when the displayed result ofrecognition is a telephone number, user confirmation of said displayedresult of recognition transitions the portable terminal device directlyto a phone state wherein a recipient phone number field is pre-populatedduring said direct transition with said lust-recognized telephonenumber.
 13. The portable terminal device according to claim 12, wherein,when a first confirmation input is detected by the portable terminaldevice, said portable terminal device initiates communications to therecognized telephone number.